Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: cache first page of jobs.getQueryResults rows #374

Merged
merged 2 commits into from
Nov 10, 2020

Conversation

tswast
Copy link
Contributor

@tswast tswast commented Nov 5, 2020

Replaces and simplifies #347 . The cache is only used if max_results, page_size, and start_index are not set.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Towards #362 🦕

@tswast tswast requested review from a team and steffnay November 5, 2020 22:44
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Nov 5, 2020
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Nov 5, 2020
@tswast
Copy link
Contributor Author

tswast commented Nov 5, 2020

@busunkim96 Could you or another Yoshi take a look at this PR? It makes some modifications to an overridden google.api_core.page_iterator and google.api_core.future.polling, so we'd like a sanity check from someone more familiar with those core libraries.

@tswast tswast mentioned this pull request Nov 6, 2020
7 tasks
Copy link
Contributor

@steffnay steffnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@steffnay steffnay merged commit 86f6a51 into googleapis:master Nov 10, 2020
tswast added a commit to tswast/python-bigquery that referenced this pull request Nov 24, 2020
When there are large result sets, fetching rows while waiting for the
query to finish can cause the API to hang indefinitely. (This may be due
to an interaction between connection timeout and API timeout.)

This reverts commit 86f6a51 (googleapis#374).
gcf-merge-on-green bot pushed a commit that referenced this pull request Nov 24, 2020
When there are large result sets, fetching rows while waiting for the
query to finish can cause the API to hang indefinitely. (This may be due
to an interaction between connection timeout and API timeout.)

This reverts commit 86f6a51 (#374).

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes googleapis/python-bigquery-pandas#343
Fixes #394 🦕
@tswast tswast deleted the issue362-getQueryResults-cache branch November 16, 2023 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants